Job Parameter

Document Version: 1.0
Last Updated: 20-04-2026

---

Bronze job parameter reference

This page documents the parameters used by the generic bronze notebooks.

Parameter groups

Wrapper routing and audit

Parameter	Required	Used by	Description
audit_table	Yes	Wrapper	Fully qualified Delta table used to store workflow audit records.
default_loader_notebook	Yes	Wrapper	Default notebook path to run for loading. Usually ./02_bronze_autoloader_generic.
loader_notebook_map_json	No	Wrapper	JSON object mapping target_table_name to an alternate loader notebook path. Use {} when not needed.

Core ingestion settings

Parameter	Required	Used by	Description
target_table_name	Yes	Both	Fully qualified destination table in catalog.schema.table format.
staging_table_name	No	Forwarded / reserved	Optional stage table name. In the uploaded active loader path this is reserved for compatibility and not actively used for writes.
source_path	Yes	Both	Landing folder path containing incoming files.
checkpoint_path	Yes	Both	Structured Streaming checkpoint directory for this ingestion.
source_format	Yes	Loader	File format for Auto Loader, for example csv, json, or parquet.
source_file_pattern	No	Loader	Optional glob filter such as cbs_bc_customer*.unl.
schema_file_path	Yes	Both	Workspace path or relative path to the schema JSON file.

File parsing options

Parameter	Required	Used by	Description
delimiter	Depends	Loader	Field separator for delimited files, commonly `
header	Depends	Loader	Whether the file contains a header row. Passed as a string like true or false.
null_value	No	Loader	String value that should be interpreted as null when parsing.
rescued_data_column	No	Loader	Column used by Auto Loader to capture unexpected data, typically _rescued_data.

Write behavior

Parameter	Required	Used by	Description
output_mode	No	Loader	Streaming output mode. Usually append.
merge_schema	No	Loader	Boolean-like string controlling Delta schema merge on write.
overwrite_schema	No	Forwarded / reserved	Present for compatibility with extended patterns. Not actively used in the uploaded live write path.
load_type	Yes	Both	Business load label such as snapshot or incremental. In the uploaded active loader path it is metadata, not branching logic.
cleanup_stage_after_finalize	No	Forwarded / reserved	Reserved for extended stage/finalize logic. Not actively used in the uploaded live path.

Business lineage and control

Parameter	Required	Used by	Description
w_run_date	Yes	Both	Logical run date, usually populated from the job trigger date.
business_keys	No	Both	Comma-separated business key columns for downstream or future watermark logic.
w_business_ts	No	Both	Business-effective timestamp for the load, often populated from job trigger timestamp.
w_source_system	No	Both	Source-system identifier such as cbs.

Databricks runtime metadata

Parameter	Required	Used by	Description
w_job_name	No	Both	Databricks job name.
w_task_name	No	Both	Databricks task name.
w_job_id	No	Both	Databricks job identifier.
w_job_run_id	No	Both	Databricks job run identifier.
w_task_run_id	No	Both	Databricks task run identifier.
w_job_trigger_type	No	Both	Trigger type such as manual or scheduled.
w_job_start_ts	No	Both	Job start timestamp in milliseconds.

Parameter value conventions

Boolean-like values

Several parameters are read as strings and converted to booleans by the notebook. Use values like:

• true
• false
• 1
• 0
• yes
• no

For consistency, prefer lowercase true and false in job parameters.

Fully qualified table names

Always provide bronze tables in Unity Catalog three-part naming:

catalog.schema.table

Relative notebook and schema paths

Paths such as ./02_bronze_autoloader_generic and ./Schemas/schema_bronze_cbs_customer.json are resolved relative to the current notebook folder.

Example parameter set

The following is a representative example for a source feed:

{
  "audit_table": "deal_dev.bronze.workflow_results",
  "default_loader_notebook": "./02_bronze_autoloader_generic",
  "loader_notebook_map_json": "{}",
  "target_table_name": "deal_dev.bronze.br_cbs_customer",
  "staging_table_name": "deal_dev.bronze._stage_br_cbs_customer",
  "source_path": "/Volumes/deal_dev/bronze/cbs_customer_and_account/",
  "checkpoint_path": "/Volumes/deal_dev/autoloader/cbs_customer_and_account/",
  "source_format": "csv",
  "source_file_pattern": "cbs_bc_customer*.unl",
  "delimiter": "|",
  "header": "false",
  "null_value": "",
  "output_mode": "append",
  "merge_schema": "true",
  "overwrite_schema": "false",
  "schema_file_path": "./Schemas/schema_bronze_cbs_customer.json",
  "rescued_data_column": "_rescued_data",
  "load_type": "snapshot",
  "w_run_date": "{{job.trigger.time.iso_date}}",
  "business_keys": "customer_id",
  "w_business_ts": "{{job.trigger.time.iso_datetime}}",
  "cleanup_stage_after_finalize": "true",
  "w_source_system": "cbs",
  "w_job_name": "{{job.name}}",
  "w_task_name": "{{task.name}}",
  "w_job_id": "{{job.id}}",
  "w_job_run_id": "{{job.run_id}}",
  "w_task_run_id": "{{task.run_id}}",
  "w_job_trigger_type": "{{job.trigger.type}}",
  "w_job_start_ts": "{{job.start_time.timestamp_ms}}"
}

Which parameters usually change for a new source

When creating a new job, these are the fields you almost always update:

• target_table_name
• staging_table_name
• source_path
• checkpoint_path
• source_file_pattern
• schema_file_path
• load_type
• business_keys
• w_source_system

You usually keep these unchanged unless the source format differs:

• default_loader_notebook
• loader_notebook_map_json
• output_mode
• rescued_data_column